Known multimedia retrieval techniques include a document retrieval technique for retrieving a document with a keyword serving as a search condition, a similar image retrieval technique for retrieving a similar image with an image serving as a search condition, and a music retrieval technique for retrieving music with humming serving as a search condition. These techniques are disclosed in the paper entitled “Use of Language Processing Technique in Information Retrieval System” contributed by Kouji KOMATSU, Keiei Ronsyu, pp. 105-114, Vol. 13, No. 1, December 2003, the paper “Video Retrieval Technique—Increasing Need thereof and Technical Innovation” authored by Takemochi ISHII, Techno Current, No. 419, Jun. 1, 2006, and the paper entitled “A Music Retrieval Method Based on Tendencies of Feature Value throughout A Music,” authored by Kazuhisa OONO, Yu SUZUKI, and Kyoji KAWAGOE, Journal of the Database Society of Japan (DBSJ), Vol. 7, No. 1, pp. 233-238, 2008.
The known multimedia retrieval techniques typically allow a media of the same type as the one input as a search condition to be retrieved. In this specification, the media collectively refer to text, image, music, etc. Any one of the media including one type of medium is referred to as single media, and any one of the media including two or more types of media is referred to as composite media. For example, a moving image including an image and music, a document including a text and an image, and a Web page including a text and an image are all referred to as composite media. Multimedia includes single media and composite media.
Methods of retrieving diverse media such as image, music, moving image with a keyword serving as a search condition are contemplated today. Retrieval of the diverse media is also referred to as cross-media retrieval. For example, a computer performing a cross-media retrieval receives a keyword input as a search condition, and extracts any Web page including the keyword through a full-text search. The cross-media retrieval computer then outputs, as the retrieval results, the extracted Web pages. The cross-media retrieval functions on the premise that images and text included on the same Web pages have similar meaning with a high probability. For example, Japanese Unexamined Patent Application Publication No. 7-105070 discloses a cross-media retrieval technique based on that premise.
In a typical retrieval process, a searcher may analyze retrieval results, perform a retrieval operation with a new search condition, and analyze retrieval results again. The searcher repeats the retrieval operation until the target information is obtained. For example, in the multimedia retrieval process, the searcher repeats the retrieval operation using an image included in the Web page resulting from the previous search operation as a search condition. In this case, the searcher downloads the image to be used as a search condition from the Web page, temporarily stores the image on the computer, and then inputs (i.e., uploads) the downloaded image as a new search condition. A history of search conditions may be managed and displayed if the number of retrieval cycles reaches a predetermined number. Such a technique is disclosed in Japanese Unexamined Patent Application Publication No. 6-301733; Degi Young and Ben Shneiderman, “A graphical filter/flow representation of Boolean queries: A prototype implementation and evaluation,” Journal of the American Society for Information Science, Vol. 44(6), pp. 327-339, 1993; Tomoyuki HANSAKI, Buntarou SHIZUKI, Kazuo MISUE, Jiro TANAKA, “Visual Search Interface for Constructing Queries while Confirming Results,” The 19th Annual Conference of the Japanese Society for Artificial Intelligence, 2005; and Anselm Spoerri, “Visual search editor for composing meta searches,” Proceedings of the 67th Annual Meeting of the American Society for Information Science and Technology, 2004.